Skip to content

Prevent data copy in VideoFrame.to_ndarray() for padded frames#2190

Merged
WyattBlue merged 4 commits intoPyAV-Org:mainfrom
lgeiger:ndarray-zerocopy
Mar 11, 2026
Merged

Prevent data copy in VideoFrame.to_ndarray() for padded frames#2190
WyattBlue merged 4 commits intoPyAV-Org:mainfrom
lgeiger:ndarray-zerocopy

Conversation

@lgeiger
Copy link
Contributor

@lgeiger lgeiger commented Mar 11, 2026

Calling VideoFrame.to_ndarray() on unaligned video frames is very slow since the frame is copied.

This PR prevents the copy for padded frames by using strided views into the frame data.

When benchmarked with the following code I'm seeing a huge speedup for padded frames. Which now has the same performance characteristics as the aligned case

import timeit
import numpy as np
import av

aligned = av.VideoFrame(1920, 1080, format="rgb24")
padded = av.VideoFrame(1080, 1920, format="rgb24")
aligned_yuv = av.VideoFrame(1920, 1080, format="yuv420p")
padded_yuv = av.VideoFrame(1080, 1920, format="yuv420p")

n = 1000
t_aligned = timeit.repeat(lambda: aligned.to_ndarray(), repeat=5, number=n)
t_padded = timeit.repeat(lambda: padded.to_ndarray(), repeat=5, number=n)
t_aligned_yuv = timeit.repeat(lambda: aligned_yuv.to_ndarray(format="rgb24"), repeat=5, number=n)
t_padded_yuv = timeit.repeat(lambda: padded_yuv.to_ndarray(format="rgb24"), repeat=5, number=n)

t_aligned = np.array(t_aligned) / n
t_padded = np.array(t_padded) / n
t_aligned_yuv = np.array(t_aligned_yuv) / n
t_padded_yuv = np.array(t_padded_yuv) / n

print(f"Aligned (1920x1080 rgb24):   {t_aligned.mean() * 1e6:.2f} ± {t_aligned.std() * 1e6:.2f} µs")
print(f"Padded  (1080x1920 rgb24):   {t_padded.mean() * 1e6:.2f} ± {t_padded.std() * 1e6:.2f} µs")
print(f"Aligned (1920x1080 yuv420p): {t_aligned_yuv.mean() * 1e6:.2f} ± {t_aligned_yuv.std() * 1e6:.2f} µs")
print(f"Padded  (1080x1920 yuv420p): {t_padded_yuv.mean() * 1e6:.2f} ± {t_padded_yuv.std() * 1e6:.2f} µs")

Before:

Aligned (1920x1080 rgb24):   1.76 ± 0.02 µs
Padded  (1080x1920 rgb24):   148.09 ± 7.51 µs
Aligned (1920x1080 yuv420p): 364.42 ± 1.05 µs
Padded  (1080x1920 yuv420p): 522.63 ± 0.08 µs

After:

Aligned (1920x1080 rgb24):   1.73 ± 0.03 µs
Padded  (1080x1920 rgb24):   1.76 ± 0.04 µs
Aligned (1920x1080 yuv420p): 365.67 ± 0.83 µs
Padded  (1080x1920 yuv420p): 368.43 ± 0.86 µs

The main changes of this PR are in e2ad2fb.

b105f6e, cd5ffd0, and 02ec746 are minor followup optimisations to ensure that this PR doesn't introduce a regression for the aligned case.

@WyattBlue WyattBlue merged commit c8356fc into PyAV-Org:main Mar 11, 2026
6 checks passed
@lgeiger lgeiger deleted the ndarray-zerocopy branch March 11, 2026 19:54
skeskinen pushed a commit to skeskinen/PyAV that referenced this pull request Mar 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants